Deformable 2d-3d registration

ABSTRACT

A method for deformable registration including determining a vector field from a two-dimensional matching of a volume of an object of interest and a two-dimensional image of the object of interest, providing a deformation profile, and finding a volume deformation that maps to a state of the two-dimensional image, wherein the deformation is parameterized by the vector field and control points of the deformation profile to find a control point configuration of the volume deformation.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims the benefit of Provisional Application No.60/957,016 filed on Aug. 21, 2007 in the United States Patent andTrademark Office, the contents of which are herein incorporated byreference in its entirety.

BACKGROUND OF THE INVENTION

1. Technical Field

The present disclosure relates to image processing, and moreparticularly to a system and method for deformable 2D-3D registration.

2. Description of Related Art

During the treatment the lesion is irradiated with high-energy beamsproduced by a linear accelerator. Treatment techniques such as 3Dconformal radiation therapy and intensity modulated radiation therapyprovide very accurate radiation to the lesion, while sparing healthytissues. Efficacy of radiation treatment planning (RTP) depends on thepatient setup at each daily fraction. The problem is to reproduce thepatient position at the time of acquiring the planning CT scans for eachfraction of the treatment process. Discrepancies between the planned anddelivered treatment positions significantly degrade the therapeuticratio.

Rigid body transformation is used to compute a correct patient setup.The drawback of such approaches is that the rigidity assumption on theimaged object is not valid for most of the patient cases, mainly due torespiratory motion.

Therefore, a need exists for a deformable 2D-3D registration.

SUMMARY OF THE INVENTION

According to an embodiment of the present disclosure, a computerimplemented method for deformable registration including determining avector field from a two-dimensional matching of a simulatedtwo-dimensional image of a volume of an object of interest and atwo-dimensional image of the object of interest, providing a deformationprofile comprising control points representing movement of the object ofinterest over time, and finding a volume deformation that maps to astate of the two-dimensional image, wherein the deformation isparameterized by the vector field and the control points of thedeformation profile to update a control point configuration of thesimulated two-dimensional image of the volume of the object of interest.

According to an embodiment of the present disclosure, a system fordeformable two-dimensional to three-dimensional registration includes adatabase for storing volumetric data of an object of interest and aprocessor for executing instructions of a plurality of modulesprocessing the volumetric data. The modules include a volume deformationmodule generating a Digitally Reconstructed Radiograph (DRR) of theimage from a deformed three-dimensional volume of the object ofinterest, a two-dimensional matching module for performing a non-rigidregistration in two-dimensions between the DDR and an actualtwo-dimensional image of the object of interest and generating atwo-dimensional vector field, and a two-dimensional to three-dimensionalback projection module, connected to the two-dimensional matching modulefor receiving the two-dimensional vector field, mapping from thetwo-dimensional vector field as a control point grid to the deformedthree-dimensional volume as a three-dimensional grid with respect to aperspective distortion.

BRIEF DESCRIPTION OF THE DRAWINGS

Preferred embodiments of the present invention will be described belowin more detail, with reference to the accompanying drawings:

FIG. 1 is a diagram of a registration framework according to anembodiment of the present disclosure;

FIGS. 2A-B are exemplary ray castings according to an embodiment of thepresent disclosure; and

FIGS. 3A-C shows an exemplary 2D control point lookup texture accordingto an embodiment of the present disclosure;

FIG. 4 shows an exemplary implementation of a back projection maskaccording to an embodiment of the present disclosure;

FIG. 5 shows an exemplary implementation of N control point masks for Nsub-cubes according to an embodiment of the present disclosure;

FIG. 6 shows an exemplary flow diagram according to an embodiment of thepresent disclosure; and

FIG. 7 is a diagram of an exemplary computer system for deformable 2D-3Dregistration according to an embodiment of the present disclosure.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

According to an embodiment of the present disclosure, using 2D-3Dregistration, a body transformation is extracted between coordinatesystems of X-ray and volumetric images, e.g., CT images. Theregistration may be used, for example, in external beam radiationtherapy, and is useful in treating cancer, where volumetric data plays arole in radiation treatment planning. A flexible framework isimplemented for deformable 2D-3D registration including a learning phaseincorporating 4D CT data sets and hardware accelerated free formDigitally Reconstructed Radiograph (DRR) generation, 2D motioncomputation, and 2D-3D back projection.

For a flexible framework for beam radiation therapy can includepre-operative 4D data and a free-form deformation based registrationscheme in 2D. Implementations in software and hardware, e.g., GPU, areconsidered.

The flexible framework finds a CT volume deformation that maps to apatients respiration state. A free-form deformation (FFD) basedframework parameterizes deformations by control points to find theoptimal control point configuration. For example, referring to FIG. 1,the framework includes a movement profile generation module 110, avolume deformation module 120, a 2D matching module 130, and a based2D-3D back projection module 140.

The movement profile generation module 110 generates deformationprofiles 111 during a learning phase from 4D CT scans 112. The movementprofile generation module 110 acquires the scans 112 at differentstages, e.g., of respiration. A registration of the scans at differentstages or time gives a displacement over time. After the registration ofthe scans 112 an obtained deformation profile 111 is normalized bytranslating 2D deformation to a control point grid configuration in 3D.

The 2D matching module 130 performs volume deformation, whereinDigitally Reconstructed Radiographs (DRRs) 131 are generated out of adeformed 3D volume 121. Here, the DRRs represent simulated 2D datagenerated from 3D reference data 112 a. The reference data used togenerate the DRR is taken from the 4D data 110.

The 2D matching module 130 performs a non-rigid registration step 132performed in 2D to match an image 133, e.g., an X-ray image, to the DDR131 resulting in a vector field 134.

The 2D-3D back projection module 140 maps a 2D control point grid 134 tothe 3D grid 111 with respect to perspective distortion to generate asimulated 3D deformation 121.

The volume deformation module 120 can implement any deformation methods,for example, volume deformation using inverse-ray-deformation (see FIGS.2A-B), GPU assisted free-form volume deformation techniques, coarse 3DBezier deformation, etc.

According to an embodiment of the present disclosure, the framework maybe implemented together with an application working with regular controlpoints, e.g., the DDR 131. For example, to achieve a deformation thatcan be used for 2D-3D deformation, the deformation performed by thevolume deformation module 120 may use a spline interpolation with asmall memory footprint and a GPU ray casting volume renderer can be usedbased on a stream model. GPU ray casting is combined with the paradigmof bended rays as shown in FIGS. 2A-B. FIG. 2A shows ray casting in adeformed volume. FIG. 2B shows deformable volume ray casting usinginverse ray deformation. FIG. 2B allows for the generation of deformedimages by bending rays in the opposite direction of the actualdeformation, without the creation of intermediate deformed volumes.

The deformation is governed by a 3D lattice of control points (see FIG.3C showing a cube 302 which symbolizes the 3D data set), dividing thevolume into sub-cubes, each of a size δ_(x), δ_(y), δ_(z), provided in aseparate input texture, organized as illustrated in FIGS. 3A-C. Theorganization of control points in a texture member is shown in FIG. 3A.A translation of control points describing the cube 302 is determinedfrom points in FIG. 3B which specify how much a corresponding point isdeformed in FIG. 3C. The sampling of the volume and the computation ofray deformation can be done in a single fragment shader with noadditional memory. To increase an accuracy of the deformation, thedisplacement of each volume element (voxel) can be determined using 3Dcubic B-splines.

An organization of 2D control point lookup texture is illustrated inFIGS. 3A-C. Each texture element stores a control point translation XYZin its RGB channels (see FIG. 3A), starting with control points atposition z=0 (a,b,c) 301.

The volume deformation method, or other method working with the regularcontrol points, outputs a 2D image. The 2D matching module 130 matchesand maps a 2D image to a 3D volume (see FIGS. 4-5), wherein the DRRserves as a moving template image and is registered to a fixed referenceimage (live X-ray), e.g., 402, following a free-form deformation scheme.The registration is formulated as an optimization of an energyfunctional aiming to minimize the distance between the two images. Sucha minimization may be determined as compared to a threshold, e.g., achange in the distance between the two images for a current iteration isless than a threshold. For integration into a registration framework,this method is ported to a graphics processor and observed largeperformance speed-up. Using back-projection masks, the vector lengths ofthe obtained 2D force field can be applied to the normalized movementprofile.

Referring to the 2D-3D back-projection 140 as illustrated in FIG. 4; amapping from a 2D control point grid 401 to a 3D grid with respect toperspective distortion is computed each time the perspective of theX-ray device changes. Generating back-projection masks 402 allows forthe detection of all control points of the free-form volume deformationthat determine the appearance of blocks in a final 2D image. For eachblock in 2D a mask 402 of a size identical to a control point texture401 is obtained as described with reference to FIGS. 3A-C. In each mask402 all control points not affecting a particular block (403) aredisabled by containing zero values (FIG. 5).

For generating the back-projection masks, e.g., 402, with given volumedimensions and a number of control points, a static control point maskfor each sub-cube can be computed during an initialization step (FIG.5).

All sub-cubes, e.g., 501, of the 3D grid 502 that affect each block in2D are determined in the same fashion as the determination of ray entrypoints and directions in the volume bounding box for GPU ray casting. Tofind out which volume sub-cubes correspond to which 2D block, e.g., 503,the method starts at a block and traverses along the ray direction in agiven step size S, e.g., S=min(δ_(x), δ_(y), δ_(z)). Initially, thefront faces F of the volume bounding box are rendered and the resultingimage is used to determine sub-cubes bordering on the bounding boxcurrent front faces. The bounding box back faces B are rendered in aseparate texture to obtain ray directions. The ray directions D=F−B andstep size S are used to obtain all remaining sub-cubes that affect thecurrent block. To obtain the back-projection masks of a particularblock, the pre-computed control point masks.(e.g., 503 of FIG. 5) of thesub-cubes that have been determined to affect the block are summed.

Referring to FIG. 6, a method for deformable registration includesdetermining a vector field from a two-dimensional matching of asimulated two-dimensional image of a volume of an object of interest anda two-dimensional image of the object of interest at block 601,providing a deformation profile comprising control points representingmovement of the object of interest over time at block 602, and finding avolume deformation that maps to a state of the two-dimensional image,wherein the deformation is parameterized by the vector field and thecontrol points of the deformation profile to update a control pointconfiguration of the simulated two-dimensional image of the volume ofthe object of interest at block 603. The vector field maps differencesbetween the simulated two-dimensional image of the volume of the objectof interest and the two-dimensional image of the object of interest.Finding the volume deformation includes a two-dimensional tothree-dimensional back projection mapping the vector field to thedeformation profile with respect to perspective distortion, andgenerating a simulated three-dimensional deformation.

It is to be understood that the present disclosure may be implemented invarious forms of hardware, software, firmware, special purposeprocessors, or a combination thereof. In one embodiment, an applicationprogram is tangibly embodied on a program storage device. Theapplication program may be uploaded to; and executed by, a machinecomprising any suitable architecture.

Referring to FIG. 7, according to an embodiment of the presentdisclosure, a computer system 701 for deformable 2D-3D registration cancomprise, inter alia, a central processing unit (CPU) 702, a memory 703and an input/output (I/O) interface 704. The computer system 701 isgenerally coupled through the I/O interface 704 to a display 705 andvarious input devices 106 such as a mouse and keyboard. The supportcircuits can include circuits such as cache, power supplies, clockcircuits, and a communications bus. The memory 703 can include randomaccess memory (RAM), read only memory (ROM), disk drive, tape drive,etc., or a combination thereof. Embodiments of the present disclosurecan be implemented as a routine 707 that is stored in memory 703 andexecuted by the CPU 702 to process the signal from the signal source708. The computer system 701 further includes a graphics processing unit(GPU) 709 for processing graphics instructions, e.g., for processing thesignal source 708 comprising image data. As such, the computer system701 is a general purpose computer system that becomes a specific purposecomputer system when executing the routine 707. The computer system 701may further include a database 710 for storing volumetric data, etc.

The computer platform 701 also includes an operating system and microinstruction code. The various processes and functions described hereinmay either be part of the micro instruction code or part of theapplication program (or a combination thereof) which is executed via theoperating system. In addition, various other peripheral devices may beconnected to the computer platform such as an additional data storagedevice and a printing device.

It is to be further understood that, because some of the constituentsystem components and method steps depicted in the accompanying figuresmay be implemented in software, the actual connections between thesystem components (or the process steps) may differ depending upon themanner in which the present invention is programmed. Given the teachingsprovided herein, one of ordinary skill in the related art will be ableto contemplate these and similar implementations or configurations ofthe present disclosure.

Having described embodiments for deformable 2D-3D registration, it isnoted that modifications and variations can be made by persons skilledin the art in light of the above teachings. It is therefore to beunderstood that changes may be made in embodiments of the presentdisclosure that are within the scope and spirit thereof.

1. A computer implemented method for deformable registration comprising:determining a vector field from a two-dimensional matching of asimulated two-dimensional image of a volume of an object of interest anda two-dimensional image of the object of interest; providing adeformation profile comprising control points representing movement ofthe object of interest over time; and finding a volume deformation thatmaps to a state of the two-dimensional image, wherein the deformation isparameterized by the vector field and the control points of thedeformation profile to update a control point configuration of thesimulated two-dimensional image of the volume of the object of interest.2. The computer implemented method of claim 1, wherein the vector fieldmaps differences between the simulated two-dimensional image of thevolume of the object of interest and the two-dimensional image of theobject of interest.
 3. The computer implemented method of claim 1,wherein finding the volume deformation comprises: a two-dimensional tothree-dimensional back projection mapping the vector field to thedeformation profile with respect to perspective distortion; andgenerating a simulated three-dimensional deformation.
 4. The computerimplemented method of claim 3, further comprising a deformation forgenerating the simulated two-dimensional image of the volume of theobject of interest from the simulated three-dimensional deformation. 5.A system for deformable two-dimensional to three-dimensionalregistration comprising: a database for storing volumetric data of anobject of interest; and a processor for executing instructions of aplurality of modules processing the volumetric data, the modulescomprising: a volume deformation module generating a DigitallyReconstructed Radiograph (DRR) of the image from a deformedthree-dimensional volume of the object of interest; a two-dimensionalmatching module for performing a non-rigid registration intwo-dimensions between the DDR and an actual two-dimensional image ofthe object of interest and generating a two-dimensional vector field;and a two-dimensional to three-dimensional back projection module,connected to the two-dimensional matching module for receiving thetwo-dimensional vector field, mapping from the two-dimensional vectorfield as a control point grid to the deformed three-dimensional volumeas a three-dimensional grid with respect to a perspective distortion. 6.The system of claim 5, further comprising a movement profile generationmodule for capturing volumetric data of the object of interest overtime.
 7. The system of claim 6, wherein the movement profile generationmodule registers the volumetric data of the object of interest over timeoutputting the deformed three-dimensional volume.
 8. The system of claim5, wherein the database stores a two-dimensional control point lookuptexture.
 10. The system of claim 9, wherein the texture includes aplurality of texture elements, each texture element storing a controlpoint translation in respective channels.
 11. A computer readable mediumembodying instructions executable by a processor to perform a method fordeformable registration comprising: determining a vector field from atwo-dimensional matching of a simulated two-dimensional image of avolume of an object of interest and a two-dimensional image of theobject of interest; providing a deformation profile comprising controlpoints representing movement of the object of interest over time; andfinding a volume deformation that maps to a state of the two-dimensionalimage, wherein the deformation is parameterized by the vector field andthe control points of the deformation profile to update a control pointconfiguration of the simulated two-dimensional image of the volume ofthe object of interest.
 12. The computer readable medium of claim 11,wherein the vector field maps differences between the simulatedtwo-dimensional image of the volume of the object of interest and thetwo-dimensional image of the object of interest.
 13. The computerreadable medium of claim 11, wherein finding the volume deformationcomprises: a two-dimensional to three-dimensional back projectionmapping the vector field to the deformation profile with respect toperspective distortion; and generating a simulated three-dimensionaldeformation.
 14. The computer readable medium of claim 13, furthercomprising a deformation for generating the simulated two-dimensionalimage of the volume of the object of interest from the simulatedthree-dimensional deformation.